We introduce a probability distribution, combined with an efficient sampling algorithm, for weights and biases of fully-connected neural networks.

Mixup refers to interpolation-based data augmentation, originally motivated as a way to go beyond empirical risk minimization (ERM). Its extensions mostly focus on the definition of interpolation and the space (input or embedding) where it takes place, while the augmentation process itself is less studied. In most methods, the number of generated examples is limited to the mini-batch size and the number of examples being interpolated is limited to two (pairs), in the input space. We make progress in this direction by introducing MultiMix, which generates an arbitrarily large number of interpolated examples beyond the mini-batch size, and interpolates the entire mini-batch in the embedding space.

To do so, our input trajectories are parametrized in consecutive sections, e.g. as consecutive piecewise ramps or splines.

Neural Information Processing Systems http://nips.cc/